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Abstract 

The Competent Group Communicator is an assessment tool designed to evaluate 
the performance of individual members who participate in task-oriented small group 
discussions. This instrument is iesigned to be used to evaluate the performance of 
students enrolled in a small group communication course, as a course placement tool, as a 
pre- and post-test of student mastery of group communication competencies, or to assist 
academic institutions determine the effectiveness of small group communication 
instruction in group communication courses. Support for six task competencies and three 
relational group communication competencies is provided. Task competencies are: (1) 
define the problem (2) analyze the problem, (3) identify crii'ria, (4) generate solutions or 
alternatives, (5) evaluate solutions or alternatives, and (6) maim<un task focus. Relational 
competencies are: (7) manage conflict, (8) maintain a supportive climate, and (9) manage 
group interaction. Their is a tenth general evaluation of overall problem-solving 
competency. Pedagogical and research support for these nine individual competencies is 
provided along with criteria for assessing student performance of each competency as 
either excellent, adequate, inadequate, or not used. A discussion of how the instrument 
was developed and a report of on-going validity and reliability tests are presented. An 
earlier version of this paper was presented at the 1994 Speech Communication 
Association Summer Conference on Communication Assessment. This paper extends 
our previous work by identifying a new format for assessing the problem-solving 
competency of members of a small group and incorporates our most recent efforts to 
develop a valid and reliable tool to assess small group communication problem solving. 
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The Competent Group Communicator: 

Assessing Essential Competencies of 
Small Group Problem Solving 

Steven A. Beebe, Southwest Texas State University 
Kevin J. Barge, Baylor University 
Colleen McCormick, University of Colorado, Colorado Springs 

Communicating in small groups has long been valued as an important skill. In 
ancient Greece Socrates used the dialectical skill of using questions to search for truth, a 
systematic process which served as the foundation for today’s college-level courses in 
small group communication. We continue to teach students how to seek answers to 
questions, solve problems and make decisions in small groups. Group communication 
skills are increasingly held in high esteem as corporate quality improvement programs, 
employee involvement efforts, and teamwork all hinge on individuals effectively 
communicating in small groups (Scholtes, 1988). The ubiquitous group meetings remains 
one of the most popular formats for making decisions and solving problems (Mosvick & 
Nelson, 1987). Yet, despite the unchallenged value of group deliberations, there have 
been few systematic effons to identify and assess the key competencies of working in 
groups. 

This paper presents a description of the on-going development and methods used 
to assess small group communication competence. The method for developing this 
instrument were modeled after the successful procedures used by the Speech 
Communication Association’s (SCA) Committee on Assessment and Testing’s (CAT) 
approach to assessing public speaking called The Competent Opeaker (Morreale, Moore, 
Taylor, Surges-Tatum, & Hulbert-Johnson, 1992). We organized our efforts to develop 
an instrument following these five steps: (1) identify the purpose of the instrument, (2) 
identify small group communication competencies, (3) identify criteria for assessing the 
competencies, (4) develop the instrument, and (5) test the instrument. The instrument 
reported here is in the process of development; this is not a final report. This paper 
describes the on-going efforts to develop and test a valid and reliable instrument to assess 
small group communication competencies. An earlier version of this paper was presented 
at the 1994 Speech Communication Association Summer Conference on Communication 
Assessment. This paper extends our previous work by identifying a new format for 
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assessing the problem-solving competency of members of a small group and includes our 
most recent efforts to develop a valid and reliable tool to assess small group 
communication problem solving. 



Instrument Purpose 

The Competent Group Communicator evaluation form (1995) on page 29 is 
designed to provide a valid and reliable tool to assess performance in a small group 
problem-solving or decision-making discussion. There are many purposes for group 
deliberation (e.g. therapy groups, study groups, decision-making groups). This 
instrument is designed to assess individual group member competency in task-oriented 
discussions in which there is a problem to solve or a decision to make. Most small group 
communication research has investigated group problem solving or decision making 
(Frey 1988). This instrument is primarily designed to assist educators evaluate student 
performance in small group discussions as typically taught in group communication 
courses presented in North America. The instrument focuses on assessing behaviors 
rather than knowledge about small group communication or motivation to participate in a 
group discussion. We agree with Hay (1994), when she calls for “alternative assessment” 
measures; these are measures which are grounded in the competencies we teach in our 
classrooms and assessed by observing students perform the behaviors. 

The instrument described here may have several applications: It may help 
educators (1) evaluate student performance in problem-solving and decision-making 
groups discussions in classes; (2) assess student’s skill tlius serving as a placement tool 
for participating in group discussions, (3) measure pre- and post-test student mastery of 
small group communication behaviors taught in a small group communication course, 
and (4) generate assessment data which could help academic institutions determine the 
effectiveness of small group communication instruction and student mastery of group 
communication skills. While it is primarily designed to assess student performance in 
academic settings, it could also be used in corporate training and development programs; 
teamwork and group problem-solving and decision-making ability remain valued skills in 
organizations. 

Identifying Small Group Communication Competencies 

Several scholars have sought to develop competencies and assessment tools for 
specific communication contexts, of which public speaking and interpersonal 
communication appear to be the most popular (Rubin, 1984). While a multitude of 
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instruments have been designed to assess various group outcomes or a specific group 
member’s behavior (e.g., McCroskey & Wright, 1971; Kaplan & Greenbaum, 1989; 
Leathers, 1969; Greenbaum, Kaplan, & Damiano, 1991), there have been comparatively 
few systematic efforts to identify and assess small group communication competencies 
which would lead to the construction of an assessment instrument. CAT (Morreale, 
Moore, Taylor, Surges-Tatum, Hulbert-Johnson, 1992) specifically identifies small group 
communication as one of the contexts for identifying competencies: “As a minimum, 
assessment should occur in the one-to-many setting (e.g., public speaking, practical small 
group discussions . . .)”. 

We reviewed three bodies of work and discussed our work with group 
communication educators as we sought to identify small group communication 
competencies. First, we reviewed the leading college-level textbooks identified through a 
survey conduced by Wamamunde (1986) as well as discussions with textbook publishers 
regarding the popularity of small group communication texts. Our goal was to identify 
common skills and competencies recommended by group communication textbook 
authors to ensure face validity of the instrument. Second, we examined research which 
has sought to identify essential small group communication behaviors which lead to 
group effectiveness. Third, we reviewed extant small group assessment tools as well as 
research which has summarized the effectiveness of existing group assessment 
instruments. Finally, we presented our initial conclusions gleaned from reviewing texts, 
research, and existing instruments to several groups of small group communication 
educators seeking their recommendations about how to improve the assessment process 
and format. 

Group Pedagogy. As Kelly and Phillips (1990) note, “If teachers arc to perform 
their pedagogical mission, they must first identify what skills are to be taught. Scattered 
through texts in small group discussion are suggestions about what students ought to do 
to be effective group members” (p. 6). A content analysis of selected small group 
communication texts identifies the key topics presented. Kerr (1990) conducted a word 
count of nine widely used small group communication texts in an effort to identify trends 
in the among of attention given to small group communication principles and skills. As 
reported in Beebe & Barge (1993) she found the most consistently covered topics include 
defining small group communication, group communication theories and models, roles, 
cohesiveness, conflict management, interpersonal communication, problem solving 
agenda and leadership. Her results arc generally consistent with the survey conducted by 
Wamemunde (1986) who found that leadership, problem-solving patterns, conflict 
management in groups, and group roles were the most frequent topics covered in small 
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group communication classes, as determined by the number of class period devoted to 
each topic. 

Most small group communication educators organize group member behavior into 
task and relationship dimensions. Behaviors associated with the group task involve 
helping the group make a decision, solve a problem or perform its assigned task. John 
Dewey’s (1910) reflective thinking sequence remains an important launch pad for most of 
the task “competencies” small group communication textbook authors recommend. 

Cohen (1994) suggests that Dwight Sheffield (1922) was one of the first persons to write 
a text on the subject of group discussion. McBumey and Hance's (1939) text ,The 
Principles and Methods of Discussion , was another pioneering work that built upon the 
work of both Dewey and Sheffield. Although other formats for organizing group 
deliberations have been developed (e.g., single question, ideal solution, RISK) (See 
VanGundy, 1981), most are derivative from the problem-to-solution sequence of 
activities Dewey identified. 

Aitken and Neer (1992), in suggesting a competency-based core curriculum for 
communication departments in higher education, offered some recommendations for 
“decision-making competence.” While not specifically explicating small group 
competence, their suggestions clearly have implications for task-oriented small group 
deliberations: 

Decision-making competence includes knowledge of: reflective thinking 
processes, rhetorical sensitivity, argumentation methods, decision emergence, 
task process activities, relational activities, topic focus, listening, critical thinking 
processes and developmental decision-making. The competent decision-maker is 
able to determine the most appropriate methods by which to communicate 
effectively, while applying various communication competencies to the decision 
making process, (p. 270). 

Taking a more holistic perspective. Ford and Wolvin (1993) identified the 
following competencies for the small group communication component of a basic hybrid 
speech communication course: 

Completing tasks in a small group situation 
Interacting with others in a small group situation 
Listening to others in a small group situation 

Feeling comfortable communicating in a small group situation (p. 220) 

Even tliough there are only a handful of efforts to identify and assess small group 
competency linked to pedagogical outcomes, at least there is a point of departure. 
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Relarionship behaviors are those that help manage attitudes and feelings group 
members have toward one another. Small group communicadon textbooks are replete 
with suggested task and relationship skills in small groups. Compared to the task 
dimensions of group interactions, very few efforts have been made to develop or identify 
a comprehensive taxonomy of small group communication relational competencies. 

Most texts discuss the importance of establishing group cohesiveness, being supportive 
rather than defensive toward others, listening to others, assuming appropriate roles and 
managing conflict. 

Group Research. Compared to the contexts of public speaking, voice and 
articulation, listening and interpersonal communication, there has been little effort to 
identify core small group problem-solving communication competencies, yet group 
problem solving and decision making constitute a central theme in small group 
communication research. Frey (1988) reports that all but 4.3% of small group 
communication research published during the 1980’s investigated problem-solving or 
decision-making groups. Research conducted by Randy Hirokawa (1985, 1988a, 1988b, 
1992) has produced the most fruitful insights in specifying group member behavior which 
leads to enhanced group performance. Identifying how communication functions to 
improve group effectiveness has resulted in the identification of several key behaviors 
that improve group outcomes. Among the key questions that the functional 
communication literature suggests group members address are; (1) Is there something in 
the present situation that needs to be changed? (2) What is the goal the group wants to 
achieve? (3) What choices does the group have that will help achieve the goal? and (4) 
What are the positive and negative implications of the choices? Answering these 
questions will result in vigilant thinking which will, in turn, produce better quality results. 
In one of the few studies to look explicitly at competency in small groups, Barge and 
Hirokawa (1989) contend that communication skills can be organized into two categories; 
(1) task and (2) relational. Task competencies are similar to the ones previously 
identified and include skills at establishing operating procedures, analyzing the problem, 
generating solutions, evaluating solutions, and implementing solutions. Relational skills 
concern the managing of the interpersonal climate and the structuring of roles within the 
group. Skills such as interaction management, expressiveness, other-orientation, and 
relaxation are designated as key skills that build constructive group climates. 

Group Assessment Tools. A variety of methods have been developed by 
educators and researchers to measure various aspects of gro . p member behavior (Bales & 
Cohen, 1979; McCroskey & Wright, 1971; Leathers, 1969; Kaplan & Greenbaum, 1989; 
Greenbaum, KapLn & Damiano, 1991). Small group communication textbooks also 
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include numerous group measurement instruments which reflect the diversity in 
measuring various aspects of group member behavior (e.g., Beebe & Masterson, 1994; 
Galanes & Brilhan, 1994). Yet in spite of these assessment efforts, Kelly, Kahn, and 
McComb (1990) suggest, “There is no hard-and-fast rule available for the evalaaiion of 
discussion outcomes . . (p. 7). Beebe and Barge (1993) reviewed group assessment 
measures found in group communication texts and found considerable variation in the 
approaches used to assess both individual group member behavior as well as group 
outcomes. Reviewing the various efforts to assess group behaviors leads to the 
unprofound conclusion that there is little agreement as to which group behaviors should 
be assessed. 

One research team systematically reviewed, categorized and evaluated 
organizational group behavior instruments. Greenbaum, Kaplan, and Damiano (1991) 
selected organizational group measurement tools published between 1950 and 1990. 
While they found almost 200 instruments only 40 were supported with reliability data. 
Further, only 19 instruments were supported by factor analysis. The researchers 
categorized the 19 instruments by level of analysis (environmental, group, individual 
member, and task) and, using a systems theory paradigm, also categorized the 
instruments by stages (input, process, output and feedback). Their research confirms that 
considerable diversity exists for measuring group communication behavior. Beebe and 
Barge (1993) summarize the research which has sought to identify group communication 
competencies and concluded, along with Greenbaum, Kaplan, and Damiano (1991) that 
there is no consensus in identifying measures of group communication. 

Identifying Group Problem-Solving Competencies 

In developing competencies fo.'’ the public speaking context, conferences have 
been convened, several committees, and individual researchers have nominated key 
competencies; from the ensuing discussion there has emerged a general consensus as to 
what constitutes essential public speaking competencies (Morreale, Moore, Taylor, 
Surges-Tatum, & Hulben- John son, 1992). In interpersonal contexts, a priori competency 
items, identified after reviewing research and pedagogy, have sometimes been presented 
to subjects for self assessment (Duran, 1983; Spitzberg, 1983; Spitzberg & Cupach, 
1984). Another method of identifying competencies is to ask subjects what they perceive 
as attributes of effective and ineffective communicators. Schrader (1989) asked 660 
university students to identify characteristics of the best and worst communicators that 
they knew. Out of a total of seventy attributes suggested, using statistical analysis, 
Schrader identified thirty-nine items that best discriminated between perceptions of 
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effective and ineffective communicators. A similar approach could be used to identify 
small group competencies. 

The competencies we identified and submitted to tests of validity and reliability 
are based upon three information sources. First and foremost, we selected the 
competencies based upon research-validated communication behaviors. Research which 
identified the function of group communication (Barge, 1990; Barge & Hirokawa, 1989; 
Hirokawa, 1885, 1988a, 1988b, 1990; Hirokawa & Rost, 1992) proved very useful in 
documenting the competencies nominated. Second, since one of the purposes of the 
assessment measure is to evaluate student mastery of competencies taught in group 
communication classes, the competencies we identify are consistent with the goals of 
small group educators. A review of group communication texts (See Beebe & Barge, 
1993) and the research by Kerr (1990) and Wamamunde (1986) were helpful in selecting 
the competencies for this instrument. Based upon existing assessment measures, we 
selected competencies valued in the workplace. Work of Kaplan and Greenbaura (1989), 
which tested the reliability of assessment instruments used to measure group 
communication behavior in the workplace, were also useful in confirming the 
competencies selected. Finally, we have also used presentations at communication 
conferences to convene groups of small group communication instructors to seek their 
assistance in designing a user-friendly tool for assessing group problem-solving 
competence. Based upon pedagogy, group research, competencies valued in applied 
setting, and discussions with potential users of the instrument, six task and three 
relationship competencies have been identified. Initially, an evaluation form, patterned 
after The Competent Speaker (Morreale, Moore, Taylor, Surges-Tatum, Hulbert-Johnson, 
1992) was the format selected; this included the list competencies and provided for 
evaluations of excellent, satisfactory, or unsatisfactory demonstration of the 
competencies. This draft of the instrument (1994) is on page 28; the mo.st recent CGC 
form (1995) in on page 29. 

Further discussion with potential users of the instrument at the 1994 SCA 
conference on communication assessment suggested that “define the problem” and 
“analyze the problem” should be two separate competencies. Potential users also valued 
a general or omnibus evaluation of a group member’s group communication problem- 
solving competency. Educators also wanted a rating sheet that could accommodate 
evaluating several group members simultaneously rather than only one group member. 
We have also sharpened our focus by explicitly considering problem-solving 
competencies rather than an assessment of group competence. The current instrument 
includes nine individual and one evaluation of overall competency competencies: 
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Group Task Competencies 

1. Deflne the problem: Appropriately defines the problem that confronts the 
group. 

2. Analyze the problem: Appropriately analyzes the problem that confronts the 
group. 

3. Identify criteria: Appropriately participates in the establishment of the group 
goal and identifies criteria for assessing the quality of the group outcome. 

4. Generate solutions: Appropriately evaluate the solutions or alternatives 
identified by group members. 

5. Evaluate solutions: Appropriately evaluates the solutions or alternatives 
identified by group members. 

6. Maintain task focus: Appropriately helps the group stay on the group task, issue 
or agenda item the group is discussing. 

Group Relational Competencies 

7. Manage conflict: Appropriately manages disagreements and conflict. 

8. Maintain climate: Appropriately provides supportive comments to other group 
members. 

9. Manage interaction: Appropriately manages interactions and invites others to 
participate. 

General Problem-Solving Competency 

10. General problem-solving skill: Uses appropriate and effective group 
communication problem solving skills. 

Table 1 presents face validity support for the nine individual competencies by 
identifying small group communication textbooks which prescribe these individual group 
member behaviors for problem-solving group discussions. Virtually all of the textbooks 
reviewed make direct or indirect references to these group task and relationship 
competencies. Table 1 also identifies research which offers support for these group 
competencies. The research cited suggests that group members who exhibit these 
behaviors contribute to group effectiveness. 

Identifying Criteria for Assessing Small Group Communication Competence 
After competencies were identified and content validity was secured by 
identifying research support and pedagogical support for the competeiijies, decisions 
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were made as to what constitutes excellent, adequate or inadequate attainmer-* of these 
competencies. The procedure for identifying criteria for competencies was similar to the 
method used in the development of The Competent Speaker (Morreale, Moore, Taylor 
Surges-Tatum, & Hulbert-Johnson, 1992). The following taxonomy identifies levels of 
excellent, adequate, inadequate, or not observed for each of the group competencies 
assessed. 



Competency One: Define the Problem 

APPROPRIATELY DEFINES THE PROBLEM THAT CONFRONTS THE 
GROUP. 

EXCELLENT 

Offers a clear, appropriate and explicit definition of the problem the group is attempting 
to solve. 

ADEQUATE 

Offers a general description of the problem the group is trying to solve. Makes at least 
one attempt to define the problem that confronts the group. 

INADEQUATE 

Inappropriately defines the problem or makes an unclear or inaccurate attempt to define 
the problem that confronts the group. Makes statements that discourage the group from 
defining the problem that confronts the group. 

DID NOT USE THIS SKILL 

Was not observed making comments which helped define the problem the group is trying 
to solve. 



Competency Two: Analyzes the Problem 

APPROPRIATELY ANALYZES THE PROBLEM THAT THE GROUP HAS 
DEFINED. 

EXCELLENT: 

Offers clear, accurate and appropriate statements which identify facts and analyze the 
causes, obstacles, history, symptoms and significance of the problem the group is 
attempting to solve. 
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ADEQUATE: 

Offers at least one statement which analyzes the causes, obstacles, history, symptoms and 
significance of the problem the group is attempting to solve. Identifies some facts about 
the problem that confronts the group. 

INADEQUATE: 

Inaccurately or inappropriately makes comments which seek to analyze the causes, 
obstacles, history, symptoms and significance of the problem the group is attempting to 
solve. Offers primarily opinions rather than facts to support his^er analysis. Makes 
statements that discourage the group from analyzing the problem. 

DID NOT USE THIS SKILL: 

Was not observed making comments which helped the group analyze the problem. 



Competency Three: Identify Criteria 

APPROPRIATELY PARTICIPATES IN THE ESTABLISHMENT OF THE 
GROUP GOAL AND IDENTIFIES CRITERIA FOR ASSESSING THE QUALITY 
OF THE GROUP OUTCOME. 

EXCELLENT 

Clearly and appropriately identifies the go^ the group is attempting to achieve and makes 
several comments which identifies or modifies appropriate criteria for an effective 
solution to the problem the group is attempting to solve. 

ADEQUATE: 

Offers at least one appropriate comment identifying the goal the group is attempting to 
achieve and makes at least one comment which identifies or modifies criteria for an 
effective solution to the problem the group is attempting to solve. 

INADEQUATE: 

Made comments which inaccurately or inappropriately identified criteria for identifying a 
solution to the problem confronting the group. Makes statements that discourage the 
group from identifying criteria. 

DID NOT USE THIS SKILL 

Offers no comments about the goal the group is attempting to achieve and makes no 
relevant comments attempting to identify or modify criteria for an effective solution the 
problem the group is attempting to solve. 

Competency Four; Generate Solutions 

APPROPRIATELY IDENTIFIES SOLUTIONS OR ALTERNATIVES TO SOLVE 
THE PROBLEM THAT THE GROUP IS SEEKING TO SOLVE. 




13 



13 



EXCELLENT 

Offers several clear, explicit and appropriate solutions or alternatives to the problem or 
decisions the group has identified and combines two or more already existing solutions to 
develop a new solution or alternative. 

ADEQUATE 

Offers at least one appropriate solution or alternative to the problem or decision the group 
has identified. 

INADEQUATE 

Offere inappropriate solutions or alternatives to the problem or decision the group has 
identified. Squelches creativity by prematurely evaluating solutions. Makes statements 
that discourage the group from identifying solutions. 

DID NOT USE THIS SKILL 

Offers no appropriate solutions or alternatives to the problem or decision the group has 
identified. 



Competency Five: Evaluate Solutions 

APPROPRIATELY EVALUATES THE SOLUTIONS OR ALTERNATIVES 
IDENTIFIED BY GROUP MEMBERS. 

EXCELLENT 

Clearly and appropriately identifies appropriate positive or negative consequences from 
solutions generated by the group sup^rted with evidence and rationale arguments. 

ADEQUATE 

Identifies at least one appropriate positive or negative consequence from the solutions 
generated by the group. 

INADEQUATE 

Makes inappropriate comments that keep the group from evaluating the solutions 
generated by the group. Makes statements that discourage the group from evaluating 
solutions. 

DID NOT USE THIS SKILL 

Makes no comments which identify positive or negative consequences from solutions 
generated by the group. 
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Competency Six: Maintain Task Focus 

A^^*>ropRIATELY helps the group stay on the group task, issue 
OR AGENDA ITEM THE GROUP IS DISCUSSING. 

EXCELLENT 

Clearly and appropriately summarizes what the group has discussed and makes 
appropriate task-relevant comments that help define the issues, analyze the issues, and 
generate or evaluate solutions. 

ADEQUATE 

Makes at least one comment that appropriately summarizes what the group has discussed 
and makes comments that are usually relevant to the group task. 

INADEQUATE 

Makes several inappropriate comments that get the group off the task at hand. Makes 
comments that are not relevant to the group’s discussions. 

DID NOT USE THIS SKILL 

Makes no attempt to summarize the group progress and frequently makes comments that 
divert the group from the task. 



Competency Seven; Manage Conflict 
APPROPRIATELY MANAGES DISAGREEMENTS AND CONFLICT. 
EXCELLENT 

Appropriately attempts to integrate differing opinions or solutions to help mange conflict 
by focusing on issues, information and evidence as opposed to personalities. 

ADEQUATE 

Makes at least one attempt to integrate differing opinions or solutions to help manage 
conflict by focusing on issues, information and evidence as opposed to personalities. 

INADEQUATE 

Makes comments to other group members which escalate conflict and contributes to 
group tension. Is insensitive the feelings of others when managing conflict. Makes 
statements which discourage the effective management of conflict. 
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DID NOT USE THIS SKILL 

Makes no attempt to integrate differing opinions or solutions to manage conflict and may 
make personal attacks on others rather than using information and evidence to manage 
conflict. 



Competency Eight: Maintain Climate 

APPROPRIATELY PROVIDES SUPPORTIVE COMMENTS TO OTHER 
GROUP MEMBERS. 

EXCELLENT 

Offers appropriate positive comments to others and/or appropriately uses humor that 
reinforces good work, manages group tension and helps maintain positive group 
relationships. e k 

ADEQUATE 

Offers at least one appropriate positive comment to another that reinforces good work 
and/or occasionally may use humor to mange group tension and maintain positive group 
relationships. Makes some effon to reinforce the good work of others. 

INADEQUATE 

Offers comments that criticize and evaluate people rather than ideas. Interjects more 
negative comments than positive comments. 

DID NOT USE THIS SKILL 

Offers no positive comments to others that reinforce good work and/or does not use 
humor to maintain a positive climate. 



Competency Nine: Manage Interaction 

APPROPRIATELY MANAGES INTERACTIONS AND INVITES OTHERS TO 
PARTICIPATE 

EXCELLENT 

Qrtisistently and appropriately initiates and terminates discussion about issues the group 
is di^ussing so as not to talk too much or too little and appropriately invites others to 
speak in an effort to distribute talk time equitably among group members. Serves as an 
effective gatekeeper in managing group interaction. 
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ADEQUATE 

Usually initiates and terminates discussion about issues the group is discussing so tis not 
to talk too much or too little and makes at least one attempt to invite others to speak in an 
effort to distribute talk time equitably among group members. 

INADEQUATE 

Consistently talks too much or too little about issues the group is discussing. 

DID NOT USE THIS SKILL 

Does not invite others to speak in an effort to distribute talk time equitably among group 
members. 



Competency Ten; General Group Problem Solving Skill 

USES APPROPRIATE AND EFFECTIVE GROUP COMMUNICATION 
PROBLEM SOLVING SKILLS 

EXCELLENT 

Overall, makes many comments that help the group defme and analyze the problem. 
Appropriately identifies criteria and generates solutions. Usually evaluates solutions and 
helps the group stay on task. Appropriately manages conflict, offers positive comments 
to maintain a positive climate and serves as a gatekeeper to manage group interaction. 

ADEQUATE 

Sometimes but not always makes comments that help the group define and analyze the 
problem. Makes some comments that appropriately identifies criteria and generates 
solutions. Sometimes makes an effort to evsiluate solutions and help the group stay on 
task. Under some circumstances manages conflict, offers some positive comments to 
maintain a positive climate and makes some effort to serve as a gatekeeper to manage 
group interaction. 

INADEQUATE 

Was detrimental to helping the group solve the problem and manage group relationships. 
Makes inappropriate comments that prohibit the group from defining and analyzing the 
problem. Makes inappropriate comments which do not help the group identify criteria or 
identify solutions. Offers inappropriate comments which did not help the group evaluate 
solutions. Made irrelevant comments and kept the group from staying on task. Did not 
appropriately conflict; often contributed to escalating conflict and group tension. Offered 
more negative comments about the group task and group members than positive 
comments. Talked too much or too little and was insensitive to other group members 
who talked too much or too little. 
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DID NOT USE THIS SKE.L 

Is not a significant contributor to the discussion. Usually is not observed making 
comments that helps the group define and analyze the problem. In general, was not 
observed making comments that appropriately identifies criteria and generates solutions. 
Was usually not observed evaluating solutions that helped the group stay on task. Was 
usually not observed managing conflict or offering positive comments to maintain a 
positive climate and was usually not observed serving as a gatekeeper to manage group 
interaction. 



Instrument Development Assumptions 

We opted for an assessment instrument which evaluates student performance 
rather than student knowledge or only self-report measures. CAT recommends that 
“speaking and listening skills must be assessed through actual performance in social 
settings (speaking before an audience, undergoing and interview, participating in a group 
discussion, etc.) appropriate to the skill(s) being assessed “ (Morrealc, Moore, Taylor, 
Surges-Tatum, & Hulbert- Johnson, 1992, p. 3). Hay (1994) also suggests that 
“alternative assessment” in speech communication has merit; alternative assessment 
procedures, as described by Herman, Aschbacher and Winters (1992) will: 

Ask students to perform, create, produce, or do something. 

Tap higher-level thinking and problem solving skills. 

Use tasks that represent meaningful instructional activities. 

Invoke real-world applications. 

[Require] People, not machines to do the score, using human judgments. 

Require new instructional and assessment roles for teachers, (p. 2). 

Assessing student performance in a problem-solving group discussion holds several 
challenges due to the interactive nature of the activity. When giving a speech one student 
can be observed at a time performing discreet behaviors. A group discussion involves 
several student performances simultaneously which results in challenges to measure 
individual student performance. 

Testing the Validity and Reliability of the Instrument 

Of paramount importance is the confirmation of the validity and reliability of ar:y 
assessment instrument. We report here the continuing procedures and results in testing 
the validity and reliability of The Competent Group Communicator (CX3C) evaluation 
form. 

A pilot test of the instrument occurred at Southwest Texas State University in 
April 1994 using an early draft of the criteria for identifying excellent, satisfactory and 
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unsatisfactoiy. A key goal of this test was to gain qualitative feedback from raters about 
the criteria and general procedures for conducting an evaluation rather than gathering 
reliability and validity data. The earlier draft of criteria was more lengthy and less 
specific that the criteria reported in this paper. A class of twenty-five students who had 
successfully completed an undergraduate course in small group communication were 
given copies of the criteria and The Competent Group Communicator evaluation form 
and were trained to evaluate group member behavior. A group of five students were 
given a case study ( The Hurricane Case”, Beebe & Masterson, 1994) and were invited to 
seek solutions to the problem confronting them in the case. Raters were trained to assess 
group member competency then observed the five-member group deliberate about the 
case study. Raters read the criteria and were given a twenty-minute lecture teaching them 
how to identify and assess the group competencies. (Group members who were being 
observed were to identify and rank-order procedures for securing their safety and 
property in the face of a rapidly approaching hurricane.) During as well as following the 
forty minute discussion, evaluators assessed group member competency using The 
Competent Group Communicator evaluation form. In this pilot study, raters were told to 
evaluate each group member on all eight competencies. After the evaluation forms were 
completed the forms were collected; raters were asked to write comments about the 
process of assessing group member behavior. The most consistent comments from raters 
indicated that the relational competencies were more difficult to assess than the taslf 
competencies. Raters also reported difficulty assessing five group member^ on eight 
competencies while observing a live group discussion. Raters would feel more 
comfortable with their evaluations if they could view the discussion on video tape so that 
they could stop, restart and review portions of the group discussion. 

Results of this first pilot test using The Competent Group Communicator form led 
to modifications in the criteria. Descriptions of the criteria were edited to enhance the 
clarity of the description and to ensure that only one type of behavior was being observed 
for each competency. Special attention was given to the rewriting the descriptions of the 
relational competencies to enhance the clarity of the behaviors assessed. 

A second pilot test of the CGC was conducted at the University of Colorado at 
Colorado Springs during the 1994 spring semester (See Appendix). In order to test the 
reliability of the instrument, McCormick (1994) attempted to replicate the psychometric 
testing model used to test the Competent Speaker Evaluation Form (Morreale, Moore, 
Taylor, Surges-Tatum, & Hulbert-Johnson, 1993). Based on the suggestions generated by 
the first pilot test conducted by Beebe, a videotape was developed by McCormick 
containing four problem-solving groups. The groups each met for approximately one and 
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one half hours and consisted of five to ten members each. The groups were actual 
problem-solving teams communicating in naturalistic settings on the campus. Next, four 
graduate teaching assistants (GTA*s) were selected to act as raters, to assess the 
individual behaviors of the individuals in the problem-solving groups on the videotape 
using the CGC evaluation form. The four raters were purposefully mixed by gender (two 
male and two female) and ethnicity (one African American, two Anglo Americans, and 
one Hispanic American). 

During a first training session, the raters were instructed to (a) review the 
standards and criteria for the competencies described on the CGC form before viewing 
the groups on the videotape, (b) view the group interaction while simultaneously 
reviewing the standards and criteria as they pertain to the individual behaviors displayed 
in the group, and (c) enter the evaluation on the rating form. The scoring system used in 
this pilot study was 1-2 (unsatisfactory), 3-4 (satisfactory), and 5-6 (excellent). 

The goals for this second pilot test were to begin to test the CGC instrument for 
interrater reliability and to conduct that testing in regard to ethnicity, age, and gender bias 
of both the rater and individual being rated. Raters met twice, for two different training 
and rating sessions, to begin to test the instrument. Rather than actually running interrater 
reliability tests, this pilot test generated suggested guidelines arid recommendations for 
such future psychometric testing of the CGC. 

As already stated, the four raters met for two different sessions of three to four 
hours each, with the intention of using the instrument to rate the small group 
communication behaviors of individuals in the groups on the videotape. During both 
sessions, raters expressed frustration ovw." the size of the groups, particularly those groups 
that consisted of more than five participants, and the length of the group interactions (one 
and one half hours) on the videotape. Based on these expressed concerns, further testing 
of the instrument should be modified accordingly. 

It was recommended that the groups to be rated should be smaller, consisting of 
approximately four to five members. Any more members than that could be described as 
a meeting (in which only one or two individuals might display a sufficient sample of the 
observable behaviors necessary for rating purposes) instead of as a small group 
discussion. Indeed, in the groups on the videotape generated by McCormick, only one or 
two individuals did dominate the communication interaction of the group. The length of 
the group interaction on the videotape also was problematic for the raters to feasibly rate 
all group members thoroughly. In a group interaction of an hour and a half in length, the 
rater is challenged in that reviewing the tape separately to rate each communicant in a 
group of five or more could take as much as seven hours. Conversely, if the raters watch 
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only a portion of the videotape for a given communicant, that individual might not 
display the observable behaviors necessary to be assessed. 

Another concern of the raters during both sessions was how to rate an individual 
if a given competency listed on the CGC was not displayed. It is paramount that the 
communicator display sufficient behavior characteristics of a given competency in order 
to use the form effectively. However, if the behavior is not exhibited by a communicator, 
some guidelines need to be provided for scoring and using the Competent Group 
Communicator form effectively. 

Suggested guidelines for the future testing of this instrument include extensivij 
training of the raters. The raters need to be clear on what behaviors represent the various 
competencies on the form. Additionally, future testing should address what may 
constitute a sufficient sample of performance such that an informed evaluation of a given 
competency can occur. 

Based on the second pilot test and the two rating sessions conducted at the 
University of Colorado at Colorado Springs, it is obvious that the effective utilization of a 
small group assessment tool may involve some difficulties to be addressed by the 
researcher. The interactive process nature of small group communication provides 
challenges for both the development and the testing of a psychometrically sound tool. Yet 
developing such an instrument can be of value as we seek to understand how to enhance 
the communication competency of individuals communicating with others in small 
groups. 

Following these initial pilot tests, the instrument was presented at the SCA 
Conference on Assessment in August, 1994. Three groups of small group 
communication instructors evaluated the instrument and made several recommendations 
that are reflected in the current form. These groups served as focus groups to assist the 
instrument development team develop a user-friendly and pedagogicaly useful tool. 
Group communication instructors wanted a form that would permit them to evaluate a 
group of at least five people at the same time. Pilot test data as well as comments from 
potential users suggest that instead of a three-point evaluation of “unsatisfactory,” 
“satisfactory,” or “excellent,” that a fourth category of “did not use this skill” should be 
added and that the word “adequate” would replace “satisfactory” and “inadequate” would 
replace “unsatisfactory.” Focus group members also valued a way of providing 
feedback not only to individual members but also to provide a general assessment of the 
group’s performance of the ten competencies. 
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Procedures for Using the Current Assessment Form 

The current version of the Competent Group Communicator is designed to be 
used by trained raters who view a group discussion on video tape so that they can review 
the discussion to ensure accuracy while evaluating the behavior of several group 
members. The suggested procedures are as follows: 

1. Raters are trained to use the CGC form. Descriptions of “Excellent,” 

“Adequate,” and “Inadequate” display of the competencies are given to the raters being 
trained to use the form and are reviewed. To assist in the training, sample videotaped 
excerpts illustrating each of the nine individual competencies should be developed and 
shown to raters during the training. Group raters should practice viewing these taped 
examples and coding them accurately. . 

2. A group of up to 5 people will be videotaped while attempting to solve a problem. 
The problem-solving scenario should last approximately 45 minutes to one hour. The 
problem should involve the use of data or evidence so that group member’s abilities to 
appropriately analyze and define the problem may be assessed. It should also be a 
problem that permits the group to generate multiple solutions to the problem. 

3. The video taped problem-solving discussion will be shown to the trained raters. 
Each rater will assess each member of the group based upon viewing the video taped 
discussion. The raters will also rate the entire group as a whole. Raters may review 
the video tape as often as they like until they believe that have accurately assessed each 
group member and the entire group. 

An Invitation 

This revised version of the instrument was tested in October 1995 at Southwest 
Texas State on a live group discussion as a pilot of the instrument included here. 

Graduate students in a Seminar in Small Group Communication class were trained 
(without video taped exemplars) to recognize the display or non-display of the nine 
individual group problem-solving competencies. Results indicated an improved ability 
to use the form yet many raters would have valued reviewing the discussion on video tape 
at least one more time to enhance their confidence in accurately assessing member 
competencies. Yet is seemed clear to both the raters and to us that viewing the 
discussion on video tape would enhance reliability. 
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Additional tests of the instrument are planned before wide-spread distribution of a 
“final” draft is prepared. The authors invite educators to assist in using and testing the 
instrument to determine if it meets their needs and to assess the validity and reliability of 
the instrument The instrument will be valuable only if small group educators find it 
helpful in assessing their students’ small group problem-solving competencies. TTiose 
interested in testing the instrument may contact: Steven Beebe, Department of Speech 
Communication, Southwest Texas State University, San Marcos, Texas 78666 (Phone: 
512-245-2165; FAX: 512-245-3138; e-mail SB03@academia.SWT.edu). 



Conclusion 

Courses in small group communication are a well established component of the 
communication curriculum. Corporate training and development programs continue to 
teach people how to solve problems, make decisions, work in committees and develop 
teamwork skills. Yet, despite the importance of learning small group commimication 
problem-solving skills, systematic efforts to identify and assess key problem-solving 
competencies have been limited. 

Many challenges have kept educators from developing a comprehensive 
assessment of small group communication problem-solving competencies. Basic issues 
as to what constitutes a group communication skill, how to identify the criteria for a 
competent small group communicator, how to identify the appropriate unit of analysis for 
group competence and challenges in selecting methods of assessing group competence 
need lo be addressed before a valid and reliable assessment instrument can be developed 
(Beebe & Barge 1994). 

Despite these obstacles, there are distinct advantages to forging ahead in the 
development on an instrument to assess small group problem-solving competencies. The 
development of an assessment tool will not be easy. The interactive, process nature of 
group communication problem solving provides challenges for the development of a 
psychometrically sound tool. Yet developing such an instrument can be of gi'eat value as 
we seek to understand how to enhance the abilities of communicating with otners in small 
groups. 
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APPENDIX 

A PILOT TEST OF THE COMPETENT GROUP COMMUNICATOR 



Overview 

This pilot study examined the Competent Group Communicator. The scale is an 
assessment instrument designed to evaluate the performance of individual members who 
participate in problem-solving small group discussions. This study is pas» of the on- 
going efforts to develop and test a valid and reliable instrument to assess small group 
communication competencies. 

This is only a preliminary pilot study of this instrument, intended to inform the 
instrument development process but not to provide psychometric supp 'it, as yet for this 
tool. The sample size is limited in terms of both raters and students/subjects. However, 
the results of this effort can provide direction and suggestions to the instrument’s 
developers in regard to scoring, criteria definition and rater training. 

Subjects 

The participants in this pilot study were nine students enrolled in a small group 
communication course at the University of Colorado at Colorado Springs. The subjects 
were three white males, three white females, two black females, and one Asian female. 
The students comprised two discussion groups. Students engaged in discussion in then- 
groups focusing on an issue/problem for approximately 20 minutes. 

Instrumentation 

The instrument being tested in this pilot study is the Competent Group 
Communicator (Beebe and Barge, 1994). The pilot test was conducted to test the 
instrument for preliminary inter-rater reliability and for any indication of ethnic or gender 
bias, as related to the subjects being rated. 

The instrument is designed with two sections; the first section contains nine 
competent group behaviors while the second section contains four areas to rate the 
individual’s overall group performance. Due to the lack of criteria based upon which to 
make decisions about overall group performance, the second section of the scale was 
omitted by raters in this pilot study. Therefore, the findings are based on the first section 
of the instrument, or the nine competent group behaviors. 

Procedures 
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Ta ping of subjects . Subjects were videotaped during a classroom “fishbowl” 
exercise, three student groups of 4-5 members participate in a discussion. This discussion 
takes place in the middle of the classroom with other class members acting as observers. 
Each group discussed an issue/problem for approximately 20 minutes. A microphone 
was placed in the center of the group and the entire discussion was audio and videotaped. 

Training of raters . Three raters, one white male, one black male, and one white 
female, were trained to evaluate the behaviors of the student son videotape. The raters 
were provided with the Group Rating Scale and criteria for the first nine competencies 
one week prior to the rating session. A videotape of one group discussion session was 
viewed specifically for training purposes. At the end of that viewing, the raters discussed 
the nine competencies as they pertained tot he group members’ observed behaviors and 
an attempt was made to establish a common understanding and interpretation of the 
criteria. 

Coding of data . Following the training period, raters viewed and coded the 
observed behaviors of two more test groups, one with five members, the other consisting 
of four group members. Raters chose to rate all members within each group at one time 
rather than viewing the tape four and five separate times. Hence, for the first group, each 
rater placed five scales in front of him/her and rated all group members. For the second 
group, each rater used four forms and rated each member. This appears to be the most 
feasible and less time consuming way to rate individual behaviors within groups, 
provided the group does not exceed 5-6 members. This process of rating also allows the 
rater(s) to see the entire group interaction rather than just one group member’s behavior. 
The videotape can be played back as many times as the rater feels necessary in order to 
make informed evaluative decisions. For the purposes and time constraints of this pilot 
test, the videotape was viewed once. Raters marked either Excellent, Adequate, 
Inadequate, or Did Not Use This Skill for each of the nine competencies. The testing 
took place during a three and one half hour period with interruptions. 

DATA ANALYSES AND RESULTS 

The data were statistically analyzed to calculate an alpha coefficient for inter-rater 
reliability for the three raters and to determine, using analyses of variance, whether the 
instrument demonstrated any biases in evaluating behaviors of students, related to gender 
or ethnicity. 

As indicated in the training of the raters, three raters, two male and one female, 
were trained and then tested for inter-rater reliability using Cronbach’s alpha to assess the 
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level at which they similarly identified the nine communication behaviors of the nine 
subjects. The raters reached a reliability level of .7978. 

Table 1 presents the results of analysis of variance of all nine subjects, by gender 
and by ethnicity. 

Table 1 

Nine Behaviors 

by gender of subjects 
by ethnicity of subjects 



Source of Variation 


dL 


Mean Square 


E 


Level of Significance 


Gender of Subjects 


1 


.500 


.957 


.373 


Ethnicity of Subjects 


2 


.833 


1.809 


.256 



As Table 1 indicates, there is no significant difference, based on the gender or 
ethnicity of the students in the two groups. That is, male and female students, and 
students of diverse ethnic backgrounds, were rated similarly by the three raters. 

DISCUSSION 

The findings presented indicated that the Competent Group Communicator is a 
reliable tool and free of gender and ethnic bias. However, as previously noted, this is 
only a preliminary pilot study limited by rater and subject/student size and is intended to 
inform the instrument development process and not provide psychometric support for the 
tool. Given a larger sample size the findings would most likely be different. 

Nevertheless, this preliminary study has generated directions and suggestions for scoring, 
criteria definition, and rater training. 

In regards to suggestions for scoring the instrument, raters found that evaluating 
all group members at one time proved to be the most expedient in terms of the 
communication behaviors, it is also advantageous to the raters to be able to play back the 
videotaped groups as often as necessary. Raters were able to assess the individual 
students communication behavior based on the first nine behaviors. The criteria 
pertaining to these nine behaviors was clear and understandable and therefore easy to 
norm by. However, raters found difficulty in using the final four item’s on the scale due 
to lack of criteria. Clearly define criteria for the final four items similar to the criteria 
established for the previous nine items would be very helpful to raters in the future. 

As indicated, raters found the criteria provided for the nine behaviors to be quite 
useful in rating the subjects/students. However, there was some discrepancy in how to 
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score the behaviors using the criteria provided. There was concern that the competencies 
contained quite a few requirements within each criteria level. With this in mind, it was 
noted that a subject can at one point in the discussion perform the same competency 
inadequately. It is suggested that a tracking of the behaviors be made prior to the final 
scoring of the behaviors. A rater would mark the number of time a subject behaved 
competently as v/ell as incompetently and then come up with a balance score of some 
kind that would reflect the categories of competent group behavior. 

This pilot student sought to test the Competent Group Communicator instrument 
for inter-rater reliability along with gender and ethnic bias. The findings suggest that the 
instrument does indicate reliability and shows no significant gender or ethnic bias. This 
is not a final testing of the instrument but only a pilot study to assist in the process of 
development. 
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The Competent Group Communication Evaluation Form (1994) 

Eight Small Group Competencies Performance Ratings 

Unsatisfactory Satisfactory Excellent 

Group Task Competencies 

Competency One ; Defines and Analyzes 
the Problem. APPROPRIATELY 
I DEFINES AND ANALYZES THE 

PROBLEM THAT CONFRONTS THE 

GROUP. 

Competency Two : Identifies criteria. 

APPROPRIATELY PARTiaPATES IN 
THE ESTABLISHMENT OF THE 
GROUP GOAL AND IDENTIFIES 

CRITERIA FOR ASSESSING THE 

QUALITY OF THE GROUP OUTCOME. 

Competency Three : Generates 
solutions. APPROPRIATELY 
EVALUATES THE SOLUTIONS OR 

ALTERNATIVES IDENTIFIED BY 

GROUP MEMBERS. 

Competency Four : Evaluates solutions. ^ 

APPROPRIATELY EVALUATES THE 
SOLUTIONS OR ALTERNATIVES 
IDENTIHED BY GROUP MEMBERS. 

Competency Five : Maintains task 
focus. APPROPRIATELY HELPS THE 
GROUP STAY ON THE TASK, ISSUE 

OR AGENDA ITEM THE GROUP IS 

DISCUSSING. 

Group Relationship Competencies ~ 

Competency Six : Manages conflict. 

APPROPRIATELY MANAGES 

DISAGREEMENTS AND CONFLICT. 

Competency Seven : Maintains climate. 

APPROPRIATELY PROVIDES 
SUPPORTIVE COMMENTS TO OTHER 

GROUP MEMBERS. 



Competency Eig ht: Manages 
interaction. HELPS MANAGE 
INTERACTION AND 
APPROPRIATELY INVITES OTHERS 
TO PARTICIPATE. 



General Comments: 



Summative Scores of Competencies: 
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